Updated ca certificate and policy packages.
Updated IGTF accredited anchors (https://dl.igtf.net/distribution/igtf/current/).
Nothing seems to work:
[rbarnsley@src-dev ~]$ davix-cp --debug -H "Authorization: Bearer $BEARER_TOKEN" https://srcdev.skatelescope.org:443/storm/sa/coral_test2 https://srcdev.skatelescope.org:443/storm/sa/coral_test5
(Davix::HttpThirdPartyCopy) Error: Transfer failure: SSLHandshakeException while pushing https://srcdev.skatelescope.org:443/storm/sa/coral_test5: The peer's certificate with subject's DN CN=srcdev.skatelescope.org was rejected. The peer's certificate status is: FAILED The following validation errors were found:In StoRM server logs:
2023-05-31T12:43:36.062Z [0ea8d9e3-bd4f-4b23-b250-ad7cb47dd399] INFO 27704 --- [thread-pool-19] o.i.storm.webdav.tpc.TransferFilter : Push third-party transfer completed: ERROR : SSLHandshakeException while pushing https://srcdev.skatelescope.org:443/storm/sa/coral_test5: The peer's certificate with subject's DN CN=srcdev.skatelescope.org was rejected. The peer's certificate status is: FAILED The following validation errors were found:
error affecting the whole chain (category: X509_CHAIN): No trusted CA certificate was found for the certificate chain
error at position 1 in chain, problematic certificate subject: CN=R3,O=Let's Encrypt,C=US (category: X509_CHAIN): Trusted issuer of this certificate was not established
error at position 0 in chain, problematic certificate subject: CN=srcdev.skatelescope.org (category: NAMESPACE): Namespace definition for the certificate issuer (CN=R3,O=Let's Encrypt,C=US) is not defined, and namespaces are configured to be required.
error at position 1 in chain, problematic certificate subject: CN=R3,O=Let's Encrypt,C=US (category: NAMESPACE): Namespace definition for the certificate issuer (CN=ISRG Root X1,O=Internet Security Research Group,C=US) is not defined, and namespaces are configured to be required.. Source: /storm/sa/coral_test2, Destination: https://srcdev.skatelescope.org:443/storm/sa/coral_test5, Bytes transferred: 0, Duration (msec): 16, Throughput: -/sec, id: 0ea8d9e3-bd4f-4b23-b250-ad7cb47dd399which seems to point towards:
[root@storm-webdav certificates]# openssl s_client -connect srcdev.skatelescope.org:443
CONNECTED(00000003)
depth=2 C = US, O = Internet Security Research Group, CN = ISRG Root X1
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = srcdev.skatelescope.org
verify return:1
---
Certificate chain
0 s:/CN=srcdev.skatelescope.org
i:/C=US/O=Let's Encrypt/CN=R3
1 s:/C=US/O=Let's Encrypt/CN=R3
i:/C=US/O=Internet Security Research Group/CN=ISRG Root X1which is included in /etc/grid-security/certificates:
[root@storm-webdav certificates]# ls -al | grep isrg
-rw-r--r-- 1 root root 1939 May 31 12:45 isrg-root-x1.pem
-rwxr-xr-x 1 root root 1939 May 31 12:45 isrgrootx1.pem
-rwxr-xr-x 1 root root 200 May 31 12:45 isrgrootx1.signing_policy
-rw-r--r-- 1 root root 1923 May 31 12:45 isrg-root-x1-xsigned.pem
-rw-r--r-- 1 root root 790 May 31 12:45 isrg-root-x2.pemand 2) namespace restrictions, which are done using signing policies to limit certificate subjects per issuer.
What makes no sense to me is why other sites using let's encrypt and StoRM, e.g. SPSRC, JPSRC, KRSRC don't seem to have this problem, even when the server is returning certificates that are signed by retired root certificates:
[root@storm-webdav certificates]# openssl s_client -connect spsrc14.iaa.csic.es:18027 -showcerts
...
2 s:/C=US/O=Internet Security Research Group/CN=ISRG Root X1
i:/O=Digital Signature Trust Co./CN=DST Root CA X3Need to look at:
Remaining bits for metadata extended client:
F2F program forum meeting.
Wrote download plan using metadata to store hierarchy.
F2F program forum meeting.
Need to test it against prototype datalake.
Wrote upload plan using metadata to store hierarchy.
Need to implement download plan equivalent.
Fixed argo (was spawning cronjobs with kind=v1beta not v1).
Demoed IVOA work.
Meetings re: rucio integration epic.
Fired off metadata sig message (gather peoples thoughts..).
Preparing for IVOA demo. Did some argo work trying to install rucio-task-manager, but didn't work.
Finished migration of server and daemons. Tested upload and transfer.
Some team catch up in morning.
(Started) migrating Rucio in afternoon.
ART Sync. Meeting with Tim from STFC about tokens on multi-vo. Automated the three bullet points from yesterday in ansible proxy machine creation.
Started ansiblifying proxy machine. Mostly done:
https://gitlab.com/ska-telescope/src/ska-src-ansible/-/tree/main#proxy-machine-proxymf
still need to:
/etc/request_cert_certbot.sh and /etc/update_certs.sh but need to make sure only letsencrypt frontend is available when this happens.Some dependency on LetsEncrypt as CA probably needs clearing up.
Tidying up README, making graphics etc.
Tidied this all up. Added access_url in DaCHS postgres init that points to datalink service and tested.
Added new notebook.
Created datalink response that populates a single result with semantic "this", pointing to storage.
Added token retrieval through auth code workflow:
response = requests.get("https://srcdev.skatelescope.org/rucio-dev/auth/oidc")
auth_url = response.headers['X-Rucio-OIDC-Auth-URL']
auth_code = input()
response = requests.get("https://srcdev.skatelescope.org/rucio-dev/auth/oidc_redirect?{}".format(auth_code), headers={"X-Rucio-Client-Fetch-Token": "True"})
access_token = response.headers['X-Rucio-Auth-Token']Made service that can talk to Rucio via a service account and get replica locations.
Need to plumb in a datalink response that gets e.g. the nearest (or random for now) replica and forms a VOTable response for the client.